Katseenseurannan sovellukset mielenkiintoisen alueen HEVC-pakkaukselle

The increase in video streaming services and video resolutions has exploded the volume of Internet video traffic. New video coding standards, such as High Efficiency Video Coding (HEVC) have been developed to mitigate this inevitable video data explosion with better compression. The aim of video coding is to reduce the video size while maintaining the best possible perceived quality. Region of Interest (ROI) encoding particularly addresses this objective by focusing on the areas that humans would pay the most attention at and encode them with higher quality than the non-ROI areas. Methods for finding the ROI, and video encoding in general, take advantage of the Human Visual System (HVS). Computational HVS models can be used for the ROI detection but all current state-of-the-art models are designed for still images. Eye tracking data can be used for creating and verifying these models, including models suitable for video, which in turn calls for a reliable way to collect eye tracking data. Eye tracking glasses allow the widest range of possible scenarios out of all eye tracking equipment. Therefore, the glasses are used in this work to collect eye tracking data from 41 different videos. The main contribution of this work is to present a real-time system using eye tracking data to enhance the perceived quality of the video. The proposed system makes use of video recorded from the scene camera of the eye tracking glasses and Kvazaar open-source HEVC encoder for video compression. The system was shown to provide better subjective quality over the native rate control algorithm of Kvazaar. The obtained results were evaluated with Eye tracking Weighted PSNR (EWPSNR) that represents the HVS better than traditional PSNR. The system is shown to achieve up to 33% bit rate reduction for the same EWPSNR and on average 5-10% reduction depending on the parameter set. Additionally, the encoding time is improved by 8-20%

uvgVenctester: Open-Source Test Automation Framework for Comprehensive Video Encoder Benchmarking

acceptedVersionPeer reviewe

Design Space Exploration of Practical VVC Encoding for Emerging Media Applications

Versatile Video Coding (VVC/H.266) is the latest video coding standard designed for a broad range of next-generation media applications. This paper explores the design space of practical VVC encoding by profiling the Fraunhofer Versatile Video Encoder (VVenC). All experiments were conducted over five 2160p video sequences and their downsampled versions under the random access (RA) condition. The exploration was performed by analyzing the rate-distortion-complexity (RDC) of the VVC block structure and coding tools. First, VVenC was profiled to provide a breakdown of coding block distribution and coding tool utilization in it. Then, the usefulness of each VVC coding tool was analyzed for its individual impact on overall RDC performance. Finally, our findings were elevated to practical implementation guidelines: the highest coding gains come with the multi type tree (MTT) structure, adaptive loop filter (ALF), cross component linear model (CCLM), and bi-directional optical flow (BDOF) coding tools, whereas multi transform selection (MTS) and affine motion estimation are the primary candidates for complexity reduction. To the best of our knowledge, this is the first work to provide a comprehensive RDC analysis for practical VVC encoding. It can serve as a basis for practical VVC encoder implementation or optimization on various computing platforms.publishedVersionPeer reviewe

RDO Candidate Selection for Maximizing Coding Efficiency in a Practical HEVC Encoder

Peer reviewe

Tailored AVX2 Transform Kernels for Versatile Video Coding

Transform coding tools play an integral part in video codecs due to their substantial impact on coding efficiency. The latest video coding standard, Versatile Video Coding (VVC), makes the most of these tools by introducing new DST7, DCT8, and non-square transforms alongside the conventional DCT2 transform. This paper proposes optimized AVX2 kernels for all these transforms to speed up VVC coding. Unlike existing solutions, our kernels are specially tailored for each VVC transform type and block size. Accelerating our open-source uvg266 VVC encoder with the proposed kernels yields up to a 1.1× speedup under all intra (AI) coding condition without any coding overhead. Our implementations make forward DCT2 and DST7/DCT8 transforms 4.0× and 6.7× as fast as their respective scalar implementations in the VTM reference encoder. They also outpace the AVX2 kernels of the practical VVenC encoder by factors of 3.0× and 2.8×. The respective speedups rise up to 5.3×, 11.1×, 3.4×, and 3.0× with inverse transforms.Peer reviewe

Katseenseurannan sovellukset mielenkiintoisen alueen HEVC-pakkaukselle

The increase in video streaming services and video resolutions has exploded the volume of Internet video traffic. New video coding standards, such as High Efficiency Video Coding (HEVC) have been developed to mitigate this inevitable video data explosion with better compression. The aim of video coding is to reduce the video size while maintaining the best possible perceived quality. Region of Interest (ROI) encoding particularly addresses this objective by focusing on the areas that humans would pay the most attention at and encode them with higher quality than the non-ROI areas. Methods for finding the ROI, and video encoding in general, take advantage of the Human Visual System (HVS). Computational HVS models can be used for the ROI detection but all current state-of-the-art models are designed for still images. Eye tracking data can be used for creating and verifying these models, including models suitable for video, which in turn calls for a reliable way to collect eye tracking data. Eye tracking glasses allow the widest range of possible scenarios out of all eye tracking equipment. Therefore, the glasses are used in this work to collect eye tracking data from 41 different videos. The main contribution of this work is to present a real-time system using eye tracking data to enhance the perceived quality of the video. The proposed system makes use of video recorded from the scene camera of the eye tracking glasses and Kvazaar open-source HEVC encoder for video compression. The system was shown to provide better subjective quality over the native rate control algorithm of Kvazaar. The obtained results were evaluated with Eye tracking Weighted PSNR (EWPSNR) that represents the HVS better than traditional PSNR. The system is shown to achieve up to 33% bit rate reduction for the same EWPSNR and on average 5-10% reduction depending on the parameter set. Additionally, the encoding time is improved by 8-20%

Hardware Deceleration of Kvazaar HEVC Encoder

High Efficiency Video Coding (HEVC) doubles the coding efficiency of the prior Advanced Video Coding (AVC) standard but tackling its huge com- plexity calls for efficient HEVC codec implementations. The recent advances in Graphics Processing Units (GPUs) have made programmable general-purpose GPUs (GPGPUs) a popular option for accelerating various video coding tools. Massively parallel GPU architectures are particularly well suited for hardware- oriented full search (FS) algorithm in HEVC integer motion estimation (IME). This paper analyzes the feasibility of a GPU-accelerated FS implementation in the practical Kvazaar open-source HEVC encoder. According to our evaluations, implementing FS on AMD Radeon RX 480 GPU makes Kvazaar 12.5 times as fast as the respective anchor implemented entirely on an Intel 8-core i7 processor. However, the obtained speed gain is lost when fast IME algorithms are put into use in the anchor. For example, executing the anchor with hexagon-based search (HEXBS) algorithm is almost two times as fast as our GPU-accelerated proposal and the benefit of GPU offloading is reduced to a slight coding gain of 1.2%. Our results show that accelerating IME on a GPU speeds up non-practical encoders due to their enormous inherent complexity but the price paid with practical en- coders tends to be too high. Conditional processing schemes of fast IME algo- rithms can be efficiently executed on processors without any substantial coding loss over that of FS. Nevertheless, we still believe there might be room for ex- ploiting GPU on IME acceleration but GPU-parallelized fast algorithms are needed to get value for additional implementation cost and power budget.acceptedVersionPeer reviewe

From HEVC to VVC : the First Development Steps of a Practical Intra Video Encoder

Versatile Video Coding (VVC/H.266) is an emerging successor to the widespread High Efficiency Video Coding (HEVC/H.265) and is shown to double the coding efficiency for the same subjective visual quality. Nevertheless, VVC still adopts the similar hybrid video coding scheme as HEVC and thereby sets the scene for reusing many HEVC coding tools and techniques as is or with minor modifications. This paper explores the feasibility of developing a practical software VVC intra encoder from our open-source Kvazaar HEVC encoder. The outcome of this work is called uvg266 VVC intra encoder that is distributed under the same permissive 3-clause BSD license as Kvazaar. uvg266 inherits the optimized coding flow of Kvazaar and all upgradable Kvazaar intra coding tools, but it also introduces basic VVC intra coding tools not available in HEVC. To the best of our knowledge, this is the first work to describe the implementation details of upgrading an HEVC encoder to a VVC encoder. The rapid development time with promising coding performance make our proposal a viable approach over the encoder development from scratch.publishedVersionPeer reviewe

Open Framework for Error-Compensated Gaze Data Collection with Eye Tracking Glasses

Eye tracking is nowadays the primary method for collecting training data for neural networks in the Human Visual System modelling. Our recommendation is to collect eye tracking data from videos with eye tracking glasses that are more affordable and applicable to diverse test conditions than conventionally used screen based eye trackers. Eye tracking glasses are prone to moving during the gaze data collection but our experiments show that the observed displacement error accumulates fairly linearly and can be compensated automatically by the proposed framework. This paper describes how our framework can be used in practice with videos up to 4K resolution. The proposed framework and the data collected during our sample experiment are made publicly available.acceptedVersionPeer reviewe

Eye-Controlled Region of Interest HEVC Encoding

This paper presents a demonstrator setup for real-time HEVC encoding with gaze-based region of interest (ROI) detection. This proof-of-concept system is built on Kvazaar open-source HEVC encoder and Pupil eye tracking glasses. The gaze data is used to extract the ROI from live video and the ROI is encoded with higher quality than non-ROI regions. This demonstration illustrates that performing HEVC encoding with non-uniform quality reduces bit rate by 40-90% and complexity by 10-35% over that of the conventional approaches with negligible to minor deterioration in subjective quality.acceptedVersionPeer reviewe